addHessian: Combining quasi-Newton method with first-order method for neural network training

First-order methods such as SGD and Adam are popularly used in training Neural networks. On the other hand, second-order have shown to better performance faster convergence despite their high computational cost by incorporating curvature information. While determine step size line search approaches, first-order achieve efficient learning devising a way adjust size. In this paper, we propose new algorithm for neural networks combining methods. We investigate effectiveness of our proposed method when combined with popular - SGD, Adagrad, Adam, through experiments using image classification problems.